synchronization. Can do time server (NTP server)8) Turn off SELinux (takes effect after reboot).
A) vi/etc/rc.local
b) Set selinux=disabled
9) Maximize the use of physical memory.
A) vi/etc/sysctl.conf add
b) vm.swappiness = 0
c) sysctl-p change takes effect
10) Close the "Transparent large page", which can cause significant performance issues.
A) perform echo never >/sys/kernel/mm/redhat_transparent_hugepage/defr
Platform: Choose Linux-genericSelect Linux-generic (glibc 2.5) (x86, 64-bit), RPM for download
I download here 5.6.34 version, if the same as I downloaded, you can use the link
Http://cdn.mysql.com//Downloads/MySQL-5.6/MySQL-5.6.34-1.linux_glibc2.5.x86_64.rpm-bundle.tar
JDBC
Hive/oozie/hue and so on will be used to MySQL, so installation of MySQL is necessary. The JDBC driver is used to connect to MySQ
Use Cloudera Manager to install Hadoop
Hadoop is composed of many different services (such as HDFS, Hive, HBase, Spark, and so on). These services also have some dependencies. If you directly download the original Apache package, it is troublesome to download multiple times and configure multiple times. As a result, some companies have customized Hadoop, such as
Create a cloudera-SCM user (with a branch, This is a Linux Command ,)
[[emailprotected]cloudera-manager-5.1.0]$sudouseradd--system--home=/opt/cloudera-manager-5.1.0/run/cloudera-scm-server--no-create-home--shell=/bin/false--comment"ClouderaSCMUser"cloudera-scm
Change server_
directory.
$ Sudo tar xzf cloudera-manager * .tar.gz-C/opt/cloudera-manager
3. Create cloudera-scm
Since Cloudera Manager and Managed Services use cloudera-scm by default, you need to create this user
$ Sudo useradd -- system -- home =/opt/
machines:SCP ~/.ssh/authorized_keys [Email protected]:~/.ssh/Now log on to other machines without a password.3 Installing JavaBecause CDH4 support JAVA7, consider CDH5 only support JAVA7, decisive on. (later MySQL also used the latest 5.6.16, later found that the tragedy, do not know which reason, so the JDK has changed to the official recommendation version, or not, and the MySQL back to the 5.1.X version, the final can be.) Personal guess JDK can still use 7, MySQL can only use 5.5, and then
!
Installing the JDBC DriverFor suselinux, download mysql-connector-java-5.1.38.tar.gz, and install:
tar zxvf mysql-connector-java-5.1.38.tar.gzcp mysql-connector-java-5.1.38/mysql-connector-java-5.1.38-bin.jar /usr/share/java/mysql-connector-java.jar
To create a DB instance for another componentWe just need to create hive and activityTo create hive as an example, log in with MySQL root user after MySQL execution:
‘ro
or download the Word document: http://download.csdn.net/download/xfg0218/9747346
about CDH and Cloudera Manager
CDH (Cloudera's distribution, including Apache Hadoop), is one of the many branches of Hadoop, built from Cloudera maintenance, based on the stable version of Apache Hadoop, and integrates many patches, Can
.rpmAfter installation, if the service MySQL start did not start successfully, reported error 2002 can not find any *.pid, then use the Mysqld_safe command to start, and then see MySQL started. Set MySQL to boot: chkconfig mysqld on This installation requires the creation of the following database (excluding Cloudera Manager's database, Cloudera Manager database with related script creation, followed by in
1.7.0_80-b15)Java HotSpot (TM) 64-bit Server VM (build 24.80-b11, Mixed mode) 6. Set up NTPAll nodes Install NTP:Yum Install NTPTo configure boot boot:Chkconfig ntpd onCheck whether the setting is successful:Chkconfig--list ntpd (2-5 for on State success)To set up synchronization:Ntpdate-u ntp.sjtu.edu.cn (clock server based on the actual environment settings, this document uses the 210.72.145.44-National Time Service Center Server IP address) 7. Install the configuration MySqlYou can use Yum
:
Start command: Service httpd start
Shutdown command: Service httpd stop restart
command: Service httpd restart
View Status: Service httpd status
3. Download cm Resource Pack
To the CM resource page: http://archive-primary.cloudera.com/cm5/repo-as-tarball/5.0.2/Download the resource bundle, and the corresponding CentOS resource bundle is:
cm5.0.2-centos6.tar.gz 11-jun-2014 18:12 517M
This is the comp
1. Save the RPM package and the necessary dependent files to Linux according to the directory structure.
Download URL: e-primary.cloudera.com/cm5/redhat/6/x86_64/cm/5.9.0/
All in Node0.
Only agents and daemons are deposited in the node1-3.
Parcel incoming Linux/opt/cloudera/parcel-repo, change permissions 755.
2. Backup
Http://www.cloudera.com/content/www/zh-CN/documentation/enterprise/5-3-x/topics/cm_ag_db
I. Related software preparation and planning
1, related software and download address:
Cloudera manager:http://archive-primary.cloudera.com/cm5/cm/5/CDH installation package Address: http://archive.cloudera.com/cdh5/parcels/latest/Java Official Download (login required): http://www.oracle.com/technetwork/java/archive-139210.htmlJava versions archive
..................................... ........................................ ........................................ ...................... 52.1 set IP ..................................... ........................................ ........................................ ...................... 52.2 set hosts ..................................... ........................................ ........................................ ................. 52.3 close the firewall ........................
/hosts2.2 Installing Cloudrea ManagerYou can install the package via the official one-click button or by Yum or RPM.Installation of the official one-click installation package is described below.This installation environment is Cnetos 7, installed on 3 machinestest165 (Cloudera Manager server)test166 (Cloudera Manager Agent)test167 (Cloudera Manager Agent)2.2.1
Installation Procedure directory1.1 download the cloudera manager 4.5.1 Free Edition installation package1.2 modify machine configurations1.3 upload cloudera-Manager-installer to the specified directory1.4 modify the permissions of clouder-Manager-instanler1.5 install cloudera Manager1.6 Go To The
We all know big data about hadoop, but various technologies will enter our field of view: spark, storm, and Impala, which cannot be reflected by us. In order to better construct Big Data projects, let's sort out the appropriate technologies for technicians, project managers, and architects to understand the relationship between various big data technologies and select the appropriate language.We can read this article with the following questions:1. What technologies does hadoop contain?2. What i
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.